Image encoding method, image decoding method, image encoding device, image decoding device, and image encoding/decoding device

ABSTRACT

A decoding method includes: predicting a current block in an image using a reference block, to generate a prediction block; and generating a reconstructed block using the prediction block, wherein the generating includes: filtering for a boundary between the reconstructed block and a decoded neighboring block, using a first filter strength set using first prediction information for prediction of the current block and second prediction information for prediction of the decoded neighboring block; filtering for the boundary using a second filter strength set without using the second prediction information of the current block and the decoded neighboring block; and switching whether to execute the second filtering, based on the boundary, wherein the first filtering is in-loop filtering in which a filtered reconstructed block is used as a reference block for another block, and the second filtering is post filtering outside the loop.

CROSS REFERENCE TO RELATED APPLICATIONS

This is a continuation application of PCT International Application No.PCT/JP2013/001272 filed on Mar. 1, 2013, designating the United Statesof America, which is based on and claims priority of U.S. PatentApplication No. 61/613,646 filed on Mar. 21, 2012. The entiredisclosures of the above-identified applications, including thespecifications, drawings and claims are incorporated herein by referencein their entirety.

FIELD

The present disclosure relates to an encoding method and a decodingmethod for increasing efficiency in parallel operation on an imagesignal.

BACKGROUND

In the conventional image encoding methods (for example, see Non-patentLiterature 1) represented by the ITU-T standards denoted as H.26x andthe ISO/IEC standards denoted as MPEG-x, it is possible to divide apicture corresponding to an image signal into units called slices andencode the whole picture on a slice-by-slice basis according to a rasterscan order on the whole picture.

The HEVC standard (for example, see Non-patent Literature 2) that hasbeen standardized as a new standard introduces a tile format which makesit possible to divide a picture corresponding to an image signal intotiles having a rectangular shape, and perform a raster scan within eachof the tiles resulting from the division. This tile format allowsdivision of the picture in the vertical direction, and thus can reducethe capacity of a memory for pixel lines.

For example, FIG. 1 shows an example where a picture is divided also inthe vertical direction into tiles enclosed by solid lines. The areasenclosed by broken lines are sub-areas further divided as slices fromthe tiles in the middle of a raster scan. FIG. 1 shows a pattern inwhich each of the tile boundaries is not across any slice boundary, andFIG. 2 shows a pattern in which each of the slice boundaries is notacross any tile boundary.

CITATION LIST Non Patent Literature [NPL 1]

-   ISO/IEC 14496-10 “MPEG-4 Part 10 Advanced Video Coding”

[NPL 2]

-   Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3    and ISO/IEC JTC1/SC29/WG11 7th Meeting: Geneva, CH, -21-30 Nov.    2011, JCTVC-G1103, “High Efficiency Video Coding (HEVC) text    specification Working Draft5”,    http://phenix.itsudparis.eu/jct/doc_end_user/documents/7_Geneva/wg11/JCTVC-G1103-v12.zip

SUMMARY Technical Problem

However, the aforementioned conventional method causes distortion atprocessing boundaries when areas divided as tiles or slices areprocessed in parallel. To suppress such distortion, boundaries aresubjected to filtering which requires additional processes and memory.

The present disclosure provides methods for realizing an encoding devicewhich suppress a boundary distortion according to an input image signaland an area resulting from division, and a decoding device capable ofdecoding a stream with less distortion.

Solution to Problem

According to an aspect of the present disclosure, an image decodingmethod for decoding an image from a bitstream on a per block basis,includes: predicting a current block included in the image using areference block different from the current block, to generate aprediction block; and generating a reconstructed block using theprediction block, wherein the generating includes: first filtering forfiltering a boundary between the reconstructed block and a decodedneighboring block neighboring the current block, using a first filterstrength which is set using first prediction information used in theprediction of the current block and second prediction information usedin prediction of the decoded neighboring block; second filtering forfiltering the boundary using a second filter strength which is setwithout using the first prediction information and the second predictioninformation; and switching whether or not to execute the secondfiltering, based on the boundary, wherein the first filtering is in-loopfiltering in a loop in which a filtered reconstructed block is used as areference block for another block, and the second filtering is postfiltering outside the loop.

These general and specific aspects may be implemented using a system, adevice, an integrated circuit, a computer program, or acomputer-readable recording medium such as a CD-ROM, or any combinationof systems, devices, integrated circuits, computer programs, orcomputer-readable recording media.

Advantageous Effects

According to an aspect of the present disclosure, an encoder side canswitch between filters one of which is to be applied to a current areaboundary, based on information included in header information. In thisway, it is possible to generate a bitstream with an enhanced imagequality. Here, no filtered image is used as a prediction image. Thus, itis possible to perform filtering in parallel with other processes, whichmakes it possible to accelerate encoding and decoding processes.

In addition, with the information included in the header information, itis possible to suppress boundary distortions in a decoded image. In thisway, this embodiment provides a further increased encoding efficiencyand thus has a highly practical value.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects, advantages and features of the disclosure willbecome apparent from the following description thereof taken inconjunction with the accompanying drawings that illustrate a specificembodiment of the present disclosure.

FIG. 1 is a diagram Illustrating an example of relationships betweentiles and slices.

FIG. 2 is a diagram Illustrating an example of relationships betweentiles and slices.

FIG. 3A is a diagram for explaining a processing order of a plurality ofareas and pixels around boundaries.

FIG. 3B is a diagram illustrating an example of a conventional structurein the case of decoding a plurality of areas in parallel.

FIG. 3C is a diagram illustrating an example of a conventional structurein the case of decoding a plurality of areas in parallel.

FIG. 4 is a flowchart showing examples of conventional filteringprocesses.

FIG. 5 is a block diagram illustrating an example of a structure of animage decoding device according to Embodiment 1.

FIG. 6 is a flowchart illustrating an example of operations by the imagedecoding device according to Embodiment 1.

FIG. 7 is a block diagram illustrating an example of a structure of animage decoding device according to Embodiment 1.

FIG. 8 is a flowchart illustrating filter strength determinationprocesses in Embodiment 1.

FIG. 9A is a diagram illustrating header information defining processeswhich are performed outside a conventional decoding loop.

FIG. 9B is a table indicating the meaning of each of the values ofconventional filter_hint_type.

FIG. 10A is a diagram illustrating an example of a data structure ofinformation indicating a boundary distortion removing method inEmbodiment 2.

FIG. 10B is a table indicating an exemplary meaning of each of thevalues of filter_hint_type in Embodiment 2.

FIG. 10C is a diagram illustrating an example of the meaning of each ofthe values of filter_hint_type in Embodiment 2.

FIG. 10D is a diagram illustrating another example of a data structureof information indicating a boundary distortion removing method inEmbodiment 2.

FIG. 10E Is a diagram Illustrating another example of a data structureof information indicating a boundary distortion removing method inEmbodiment 2.

FIG. 11A is a diagram illustrating an example of a data structure ofinformation indicating a boundary distortion removing method in the casewhere an extension flag is used in Embodiment 2.

FIG. 11B is a diagram illustrating an example of the meaning of each ofthe values of across_filter_type in Embodiment 2.

FIG. 11C is a diagram illustrating another example of a data structureof information indicating a boundary distortion removing method in thecase where an extension flag is used in Embodiment 2.

FIG. 12 is a flowchart illustrating an exemplary image decoding methodin Embodiment 2.

FIG. 13 is a block diagram illustrating an example of a structure of animage encoding device according to Embodiment 3.

FIG. 14 is a flowchart illustrating an exemplary image encoding methodin Embodiment 3.

FIG. 15 shows an overall configuration of a content providing system forimplementing content distribution services.

FIG. 16 shows an overall configuration of a digital broadcasting system.

FIG. 17 shows a block diagram illustrating an example of a configurationof a television.

FIG. 18 shows a block diagram illustrating an example of a configurationof an information reproducing/recording unit that reads and writesinformation from and on a recording medium that is an optical disk.

FIG. 19 shows an example of a configuration of a recording medium thatis an optical disk.

FIG. 20A shows an example of a cellular phone.

FIG. 20B is a block diagram showing an example of a configuration of acellular phone.

FIG. 21 illustrates a structure of multiplexed data.

FIG. 22 schematically shows how each stream is multiplexed inmultiplexed data.

FIG. 23 shows how a video stream is stored in a stream of PES packets inmore detail.

FIG. 24 shows a structure of TS packets and source packets in themultiplexed data.

FIG. 25 shows a data structure of a PMT.

FIG. 26 shows an internal structure of multiplexed data information.

FIG. 27 shows an internal structure of stream attribute information.

FIG. 28 shows steps for identifying video data.

FIG. 29 shows an example of a configuration of an integrated circuit forimplementing the moving picture coding method according to each ofembodiments.

FIG. 30 shows a configuration for switching between driving frequencies.

FIG. 31 shows steps for identifying video data and switching betweendriving frequencies.

FIG. 32 shows an example of a look-up table in which video datastandards are associated with driving frequencies.

FIG. 33A is a diagram showing an example of a configuration for sharinga module of a signal processing unit.

FIG. 33B is a diagram showing another example of a configuration forsharing a module of the signal processing unit.

DESCRIPTION OF EMBODIMENTS Outline of the Present Disclosure

An image decoding method according to an aspect of the presentdisclosure is an image decoding method for decoding a bitstream havinginformation for dividing a spatial area of an image, the image decodingmethod including: obtaining, from the bitstream, a coefficient and aparameter necessary for decoding, and generating a difference value byperforming inverse quantization and inverse transform on thecoefficient; performing prediction using an encoded pixel value;generating a decoded image by adding a result of the prediction and thedifference value; filtering pixel values of the decoded image; andswitching between methods for the filtering, wherein, in the switching,a switch is made between the methods for the filtering, based on theinformation for dividing the spatial area.

For example, in the performing of prediction, the prediction may beperformed without using a result of the filtering.

For example, in the switching, information indicating the switch betweenthe methods for the filtering may further be obtained from thebitstream.

For example, in the switching, the switch between the methods for thefiltering may be made on a per division boundary basis.

For example, in the switching, the switch may be made to a different oneof the filtering methods, depending on a horizontal direction or avertical direction of a division boundary.

For example, predetermined filtering performed on a division boundarymay be only filtering for reducing block noise.

For example, predetermined filtering performed on a division boundarymay include filtering using only pixel information, in addition tofiltering identical to filtering performed on a picture boundary.

An image encoding method according to an aspect of the presentdisclosure is an image encoding method for dividing an image signal intospatial areas and encoding each of the spatial areas, the image encodingmethod including: generating a prediction signal of the image signal;obtaining a difference signal between the image signal and theprediction signal; transforming and quantizing the difference signal toobtain transform coefficient information; performing variable lengthencoding on the transform coefficient information; and performing localdecoding using the transform coefficient information to generate areconstructed image, wherein, in the performing of variable lengthencoding, information indicating a switch between methods for filteringis recorded onto a bitstream, based on information for dividing thespatial area of the reconstructed image at a decoder side.

For example, the methods for filtering may be of one or more kinds, andmethods for predetermined filtering to be performed on a divisionboundary of the reconstructed image may include a skip of filtering.

For example, predetermined filtering to be performed on a divisionboundary may be of one or more kinds, and in the performing of variablelength encoding, when a video signal is across a division boundary,information indicating that no filtering is performed on the divisionboundary may be recorded onto the bitstream.

For example, in the performing of variable length encoding, informationindicating that predetermined filtering to be performed on the divisionboundary is only filtering for reducing block noise may be recorded ontothe bitstream.

For example, in the performing of variable length encoding, informationindicating that filtering for a division boundary of a colour signal isdifferentiated from filtering for a division boundary of an other signalmay be recorded onto the bitstream.

These general and specific aspects may be implemented using a system, adevice, an integrated circuit, a computer program, or acomputer-readable recording medium such as a CD-ROM, or any combinationof systems, devices, integrated circuits, computer programs, orcomputer-readable recording media.

Hereinafter, Embodiments are described with reference to the drawings.

It is to be noted that each of the embodiments described below indicatesa general or specific example. The numerical values, shapes, materials,elements, the arrangement and connection of the elements, steps, theprocessing order of the steps etc. indicated in the following exemplaryembodiments are mere examples, and therefore do not limit the scope ofthe present disclosure. Therefore, among the structural elements in thefollowing embodiments, elements not recited in any one of theindependent claims which define the most generic concept are describedas arbitrary elements.

Embodiment 1

Embodiment 1 describes how the decoding device capable of performing theparallel decoding performs the method of decoding a bitstream inparallel and removing a boundary distortion which occurs when the imagesignals processed in parallel are reconstructed.

First, a description is given of a case of a structure in which a sliceheader is always included in the starting point of each of tileboundaries as illustrated in FIG. 1. Since tiles can be obtained byevenly dividing a picture into rectangular shapes, in an example case ofan image which has a short top boundary and is large in the horizontaldirection (an image having a resolution called 4K or 8K), it isadvantageously possible to reduce the required memory size, it is easyto evenly divide the pixel values into tile areas, and therefore it ispossible to perform parallel processing. For this reason, it isimportant to start such parallel decoding with the starting points oftiles in order to efficiently perform the parallel decoding. In thiscase, the boundaries that should be the targets from which distortion isremoved are slice boundaries.

Next, a description is given of a case of a structure in which aplurality of tiles are included in a slice as illustrated in FIG. 2.Since tiles can be obtained by evenly dividing a picture intorectangular shapes, in an example case of an image which has a short topboundary and is large in the horizontal direction (an image having aresolution called 4K or 8K), it is advantageously possible to reduce therequired memory size, it is easy to evenly divide the pixel values intotile areas, and therefore it is possible to perform parallel processing.In this case, the boundaries that should be the targets from whichdistortion is removed are tile boundaries.

FIGS. 3A to 3C are schematic diagrams for explaining a conventionaldecoding method in the case where a video having the structure in FIG. 1is decoded. FIG. 3A is an illustration for explaining in detail theorder of processing the areas A, B, C, and D, and the pixels located atthe boundary areas. The boundary areas (4 pixels×4 pixels) illustratedhere is an area on which a boundary distortion removal process isperformed. FIG. 3B is a block diagram illustrating an example of aconventional structure of a decoding device in the case where the areasA, B, C, and D are decoded in parallel. This decoding device includesdecoders DEC 1, DEC 2, DEC 3, and DEC 4, a frame memory FrameMemory fortemporality storing results of decoding, a storage unit for storinginformation (such as prediction modes, motion vectors (MVs), presence orabsence of coefficients, quantization parameters QP) necessary forfiltering, and a filtering unit. Operations are described with referenceto FIG. 4.

A bitstream BS is obtained, and area information is obtained therefrom(S401). The area n is decoded by a corresponding one of the decodersbased on area information (S402), and filtering information related to acorresponding one of hatched areas (a corresponding tile boundary area)is stored in the storage unit (S403). On the other hand, pixelInformation of the area n is stored in the frame memory for temporarystorage (S404). When not all the areas are yet decoded (NO in S405),decoding for the remaining area(s) is continued. On the other hand, whenall the areas are already decoded (YES in S405), the filtering unitfilters the pixels obtained from the frame memory based on theinformation obtained from the storage unit (S406), and outputs a decodedimage OUT.

Here, the decoders DECs 1 to 4 may be realized as a single DECprocessing unit. The structure in this example is FIG. 3C. The resultsare the same in both the cases. The storage unit and the filtering unitperform the same processes as performed by the equivalent elements inFIG. 3B, and thus no detailed descriptions are repeated.

In the above-described conventional structure, a filter process usinginformation in the storage unit is further required after the decodingprocesses performed by the DECs 1 to 4, and a filtered video is alsorequired when generating a prediction image for a next frame. Thus,high-speed processing is required, which necessitates improvements suchas an increase in the circuit scale and an increase in operationfrequency.

The decoding method in this embodiment is different in the point of notusing, for prediction, any video subjected to a distortion removalprocess.

FIG. 5 is a block diagram Illustrating an example of a structure of animage decoding device according to Embodiment 1. This decoding deviceincludes decoders DEC 1, DEC 2, DEC 3, and DEC 4, a frame memoryFrameMemory for temporality storing results of decoding, a switchinformation obtaining unit which switches filtering processes accordingto bitstream information. A flow of operations performed by thisstructure is described with reference to FIG. 6.

A bitstream BS is obtained, and area information (information fordividing a spatial area) is obtained therefrom (S601). The area n isdecoded by a corresponding one of the decoders based on area information(S602), and pixel information about the area n is stored in the framememory for temporary storage (S603). When not all the areas are yetdecoded (NO in S604), decoding for the remaining area(s) is continued.On the other hand, when all the areas are already decoded (YES in S604),the decoding image information in the frame memory is used forprediction for a next frame etc., a transition is made to a next processfor the next frame in the decoding. On the other hand, as a step ofgenerating an output image, the switch information obtaining unit nextdetermines whether or not to apply a filter on a target boundary basedon the switch information (information indicating a switch betweenfiltering methods) obtained from the bitstream BS (S605). When a filteris applied (YES in S605), the filtering unit filters the pixels obtainedfrom the frame memory, based on the filter information (or predeterminedstrength information etc.) obtained from the bitstream (S607). On theother hand, when no boundary filter is applied (NO in S605), nofiltering is performed. When not all the boundaries are yet subjected toboundary processing (execution/skip of filtering) (NO in S608), nextboundary processing is performed. When all the boundaries are alreadysubjected to boundary processing (YES in S608), a decoded image OUT isoutput.

In this way, it is possible to eliminate the need to mount the storageunit for storing information of pixels around each boundary, to reducethe scale of a circuit. With this structure, it is possible to decodefilter switch information (strength switch information) only when afilter is applied, reducing the amount of information necessary fordecoding, and to apply a filter on each boundary to be filtered withconsideration of image quality. In addition, unfiltered content in theframe memory is used for prediction, which allows, for a next frame, adecoding process to be started, resulting in an increase in operationspeed.

FIG. 7 is a block diagram illustrating a structure obtained by adding aboundary processing unit 205 to a conventional decoding device. An inputbitstream is divided into processing units, and the processing units aresubjected to decoding, inverse quantization, and inverse transform.Prediction pixel values in each processing unit are calculated by either(i) an intra predicting unit in the case of intra encoding withreference to pixels stored in the frame memory 201, specifically, pixelsof the picture including a target block or pixels of a picture notincluding the target block, or (ii) an inter predicting unit in the caseof inter encoding. The prediction pixel values are added to differencevalues obtained through inverse transform, resulting in reconstructedpixels. These reconstructed pixels are filtered in a DBF unit 202, a SAOunit 203, and an ALF unit 204, and then are stored in a frame memory 201and are output. At this time, the Deblocking filter (DBF), the SampleAdaptive Offset (SAO), and the Adaptive Loop Filter (ALF) may notexecute their processing.

Here, the DBF unit 202 executes a DBF process defined in the “Deblockingfilter process” in section 8.7 of Non-patent Literature 1 or a processdefined in the “Deblocking filter process” in section 8.7.1 in the HEVC(Non-patent Literature 2). The DBF process requires not only processingtarget pixels but also encoding modes of blocks to which the pixelsbelong, motion vectors (MVs), presence or absence of coefficients inquantization, quantization parameters (QPs) etc. as information infiltering, and selects and applies one of filter strengths from threelevels of Strong, Weak2pel, and Weak1pel. FIG. 8 illustrates a filterstrength determination flow in DBF processes.

In addition, the SAO unit 203 executes a SAO process defined in “SampleAdaptive Offset process” in section 8.7.2 in the HEVC (Non-patentLiterature 2). In addition, the ALF unit 204 executes an ALF processdefined in the “Adaptive Loop Filter process” in section 8.7.2. The SAOprocessing does not require information such as encoding modes etc. ofblocks to which target pixels belong, but requires, as information inprocessing, SAO offset information about the positions of the targetpixels and the pixels neighboring to the target pixels. The boundaryprocessing unit 205 in this embodiment rewrites only images to be outputseparately from decoded videos in the frame memory 201. Here, the filterswitch information obtained from the bitstream is separately input fromthe bitstream.

In this way, the image decoding device and the image decoding methodaccording to this embodiment make it possible to perform decodingcorrectly.

This structure makes it possible to control filtering processes suitablyfor videos, and thus to thereby suppress decrease in the image quality.

Embodiment 2

Embodiment 2 describes a data structure of information indicating aboundary distortion removing method. First, FIG. 9A illustrates headerinformation defining processes performed outside a decoding loop. Theheader information is referred to as supplemental information set (SEI)defined irrespective of conventional boundaries. It is assumed thatthese parameters indicate operations conforming to the method describedin Non-patent Literature 2 unless otherwise explained. FIG. 9B is atable indicating the meaning of each of values (Value) offilter_hint_type in the structure in FIG. 9A. In the case where thevalue is 3, it is indicated here that the value is reserved as areserved value for future use. Here, cases of the other values are thesame as in Non-patent Literature 2.

With reference to FIG. 10A, a description is given of a data structureof information indicating the boundary distortion removing method inthis embodiment.

FIG. 10A is an example of information indicating the boundary distortionremoving method in the case of maintaining the conventional datastructure with extension using the above-described reserved value forboundary filtering. The parameters illustrated in FIG. 9A and for whichno specific descriptions are given indicate operations conforming to themethod described in Non-patent Literature 2 unless otherwise explained.

Here, FIG. 10B indicates the meaning of each of the values ofconventional filter_hint_type defined newly. In the case where the valueof filter_hint_type is 3, that is, the SEI information indicates afilter for a boundary between tiles or slices, a parameter indicatingwhich one of a tile boundary and a slice boundary each having a value ofacross_filter_type is to be filtered is encoded and/or decoded. This canbe defined, for example, as a combination indicated in FIG. 10C. Whenthe value of across_filter_type is 0, the zero value indicates that nofilter is applied to the boundaries, and is not normally used (for thisreason, the value may be used as a reserved value for future extension).

The arrangement of the values 0 to 3 does not have any special meaning,and thus any other arrangement is possible. For example, the structurein FIG. 10D is possible. In this case, which one of the slice boundaryand the tile boundary is to be subjected to filtering is switched byeither across_tile_filter_flag or across_slice_filter_flag. In addition,a relationship in which when one of the values is 0, the other value is1 is known (from the value of filter_hint_type that is a superiorparameter indicating a boundary filter), one of the flags may beomitted. It is also good to arrange the flags in parallel instead ofplacing an if condition, with consideration of high-speed decoding.Here, the order of the parameters may be reversed.

In addition, in the conventional data structure, it is possible totransmit three kinds of filters depending on colour_component that iscolour information. As for boundary filtering, a less number of areasare to be filtered when the value of the above-describedfilter_hint_type is not 3, compared to the case in which all of thepixels are filtered. Here, any low-pass filter for removing boundarydistortion is possible. Thus, in many cases, it is not advantageous tochange a filter on a colour information item basis because the amount ofheader information is increased.

For this reason, a data structure as illustrated in FIG. 10E may beemployed. This is an example where the number N of filters is modifiedbased on colour_filter_flag. When this flag is not defined, the numberof filters is defined as N=3.

Here, a different filter is defined for each colour assuming that N is 3when the value of colour_filter_flag is 1, while the same filter isdefined for all of the filters assuming that N is 1 when the value ofcolour_filter_flag is 0. Alternatively, the amount of filter informationmay be reduced by switching between two kinds of signals that are aluminance signal and a colour signal assuming that N is 2 when the valueof colour_filter_flag is 1, or colour_filter_flag may be defined as avalue of N instead of the flag. A feature here is to define a structurefor controlling the number of filters.

In each of FIGS. 10A to 10E, a case where a reserved value offilter_hint_type is used. However, the number of filters may be definedusing another kind of extension flag called additional_extension_flag.An example in this case is described with reference to FIGS. 11A to 11C.Here, a method for transmitting filter coefficients defined in FIGS. 10Ato 10E may be defined in each of cases where additional_extension_flagis 1. This case can be explained in the same manner as the descriptionsgiven with reference to FIGS. 10A to 10E, and thus the same descriptionsare not repeated. In FIG. 11A, when an extension flagadditional_extension_flag is 1, the above-described across_filter_typemay be defined as in FIG. 11B (as a separate flag as in the casesillustrated in FIGS. 10A to 10E), and parameters beta_offset_div2 andtc_offset_div2 each indicating a deblocking filter strength may beencoded. As for the definitions of these parameters are the same as inNon-patent Literature 1. Here, quantization parameters for use indetermination of deblocking filter strengths may be representativevalues of tiles and slices, or QP values for deblocking filtering may benewly encoded.

FIG. 11C is an example indicating whether or not to apply a filter toeach tile boundary when tile boundary filtering is performed.Specifically, this example indicates that a filter is applied to a rightneighbouring boundary when a flagloop_filter_v_across_tiles_enabled_flag indicates 1, and no filter isapplied thereto when the flag indicates 0. Likewise, this examplefurther indicates that a filter is applied to a neighbouring boundarybelow when a flag loop_fllter_h_across_tiles_enabled_flag indicates 1,and no filter is applied thereto when the flag indicates 0.

By encoding information indicating whether or not to apply a filter toeach boundary in this way, it is possible to enhance the image quality.

An operation example according to a decoding method using a datastructure in this embodiment is described with reference to FIG. 12.

First, SEI information is obtained (S1201). Next, whether or not the SEIinformation indicates filtering on tile or slice boundary is determined(S1202). For example, a determining method here is determined based onfilter_hint_type in the cases of FIGS. 10A to 10E, and based onadditional_extension_flag in the cases of FIGS. 11A to 11C. When the SEIinformation indicates conventional post filtering on all pixels (No inS1202), filtering is performed according to the method described inNon-patent Literature 1 or the like (S1203). When the SEI informationIndicates filtering on a tile or slice boundary (YES in S1202),necessary filtering information is obtained (S1204). Here, examples ofnecessary filtering information includes: the number of filters (aswitch according to a colour signal), which one of a tile boundary and aslice boundary is a filtering target, etc. in the cases of FIGS. 10A to10E; and information for determining a deblocking filtering strength,quantization parameter information, a tile position, a filtering targetboundary, etc. in the cases of FIGS. 11A to 11C. After the decodingdevice obtains a decoded image (S1205), filtering is performed on aboundary based on the obtained information (S1206). In this way, it ispossible to enhance the image quality of the decoded image.

FIGS. 10A to 10E and FIGS. 11A to 11C Illustrate different sets ofnecessary filtering information, but the sets are non-limiting examples.A decoded image is subjected to post filtering if the sets arearbitrarily combined, which also enhances the image quality.

Here, as a non-limiting example, some information is defined in additionto already defined post_fllter_hint SEI. Alternatively, for example, areserved value selected under an if condition is extracted andseparately defined as post_across_filter SEI. In this way, the usage ofSEI can be clearly defined with a clear object defined by an applicationin the future, which may simplify an encoding device and a decodingdevice.

In this way, the use of the data structure in this embodiment indicatedin the encoding side makes it possible to enhance the image quality ofan output image, and furthermore to reduce the circuit scale of thedecoding device.

Embodiment 3

Embodiment 3 describes an encoding device which generates a bitstreamhaving the above-described data structure.

FIG. 13 is a block diagram illustrating an example of an encoding deviceaccording to Embodiment 3. An input target image signal is processed ona per coding unit basis. Prediction pixel values in each coding unit arecalculated by either (i) an intra predicting unit in the case of intraencoding with reference to pixels stored in a frame memory 101,specifically, pixels of the picture including a target block or pixelsof a picture not including the target block, or (ii) an inter predictingunit in the case of inter encoding. The differences from the inputsignals are subjected to transform, quantization, and encoding,resulting in a bitstream. In addition, the quantized coefficients aresubjected to inverse quantization and inverse transform, and then addedto the difference values, resulting in reconstructed pixels. Thereconstructed pixels are then subjected to filtering in a DBF unit 102,an SAO unit 103, and an ALF unit 104, and are stored in a frame memory101. At this time, processing by the DBF, SAO, and ALF units are notalways performed.

Here, the DBF unit 102 as illustrated in FIG. 13 executes a processdefined in the “Deblocking filter process” in section 8.7 of Non-patentLiterature 2 or a process defined in the “Deblocking filter process” insection 8.7.1 in the HEVC (Non-patent Literature 2). The DBF processrequires not only processing target pixels but also encoding modes ofblocks to which the pixels belong, motion vectors (MVs), presence orabsence of coefficients in quantization, quantization parameters (QPs)etc. as information in filtering, and selects and applies one of filterstrengths from three levels of Strong, Weak2pel, and Weak1pel. FIG. 8illustrates a filter strength determination flow in a conventional DBFprocess.

In addition, the SAO unit 103 executes a SAO process defined in “SampleAdaptive Offset process” in section 8.7.2 in the HEVC (Non-patentLiterature 2), and the ALF unit 104 executes an ALF process defined in“Adaptive Loop Filter process” in the same section 8.7.2. The SAOprocess does not require information such as encoding modes etc. ofblocks to which target pixels belong, but requires, as Information infiltering, SAO offset Information about the positions of the targetpixels and the pixels adjacent to the target pixels. It is to be notedthat the encoding side does not need to remove distortion from an imagesignal with boundary distortions to be stored in the frame memory, andthus does not include any boundary filtering unit. A boundary filteringprocess in this embodiment is executed by the encoder.

Here, an operation in a case of determining a filtering process isexplained with reference to FIG. 14.

Tile and/or slice shape information determined using a predeterminedmethod is obtained (S1401). Next, among the filter information itemsindicated in Embodiment 2, one filter information item to be encoded isdetermined (S1402). In addition, a filtering method (ON/OFF, a filtercoefficient, or the like) is determined (S1403), and filtering relatedInformation (the above information) Is entropy-encoded and recorded assupplemental information in a bitstream by the encoder in FIG. 12.

In order to further enhance the image quality of the decoded image,filter information and filtering method may be determined so that a costfunction C becomes smaller, which defined by C=D+λR where D is adifference value between an input image (an original signal before beingencoded) and a filtered image, R is the amount of codes necessary forsupplemental information, and λ is a Lagrange coefficient.

Here, the cost function C may leave a subjective distortion. Thus, thefilter information and the filtering method may be determined mainly inorder to increase smoothness of a target area and surrounding pixelareas (make these pixel areas more smooth).

This makes it possible to switch filters depending on videos, andthereby to suppress decrease in the image quality and to enhance theimage quality.

The image encoding device and the image decoding device according to oneor more aspects have been described above based on the embodiments.However, the present invention is not limited to these embodiments.Various modifications arrived at and made to the embodiments by thoseskilled in the art and embodiments configured by combining some elementsof different embodiments may also be included in the scope of the one ormore aspects, without materially departing from the scope of the presentdisclosure.

Each of the elements in the embodiments may be implemented as exclusivehardware or realized by executing a software program suitable for theelement. Each of the elements may be realized by means of a programexecuting unit such as a CPU or a processor reading a software programrecorded on a recording medium such as a hard disk or a semiconductormemory and executing the read software program. Here, the software itemsfor implementing the image decoding device and the image encodingdevice, or the like in the respective embodiments are programs below.

One program causes a computer to execute an image decoding methodaccording to an aspect of the present disclosure is an image decodingmethod for decoding a bitstream having information for dividing aspatial area of an image, the image decoding method including:obtaining, from the bitstream, a coefficient and a parameter necessaryfor decoding, and generating a difference value by performing inversequantization and inverse transform on the coefficient; performingprediction using an encoded pixel value; generating a decoded image byadding a result of the prediction and the difference value; filteringpixel values of the decoded image; and switching between methods for thefiltering, wherein, in the switching, a switch is made between themethods for the filtering, based on the Information for dividing thespatial area.

Another program causes a computer to execute an image encoding methodaccording to an aspect of the present disclosure is an image encodingmethod for dividing an image signal into spatial areas and encoding eachof the spatial areas, the image encoding method including: generating aprediction signal of the image signal; obtaining a difference signalbetween the image signal and the prediction signal; transforming andquantizing the difference signal to obtain transform coefficientinformation; performing variable length encoding on the transformcoefficient information; and performing local decoding using thetransform coefficient information to generate a reconstructed image,wherein, in the performing of variable length encoding, informationindicating a switch between methods for filtering is recorded onto abitstream, based on information for dividing the spatial area of thereconstructed image at a decoder side.

Embodiment 4

The processing described in each of embodiments can be simplyimplemented in an independent computer system, by recording, in arecording medium, one or more programs for implementing theconfigurations of the moving picture encoding method (image encodingmethod) and the moving picture decoding method (image decoding method)described in each of embodiments. The recording media may be anyrecording media as long as the program can be recorded, such as amagnetic disk, an optical disk, a magnetic optical disk, an IC card, anda semiconductor memory.

Hereinafter, the applications to the moving picture encoding method(Image encoding method) and the moving picture decoding method (imagedecoding method) described in each of embodiments and systems usingthereof will be described. The system has a feature of having an imagecoding apparatus that includes an image encoding apparatus using theimage encoding method and an image decoding apparatus using the imagedecoding method. Other configurations in the system can be changed asappropriate depending on the cases.

FIG. 15 illustrates an overall configuration of a content providingsystem ex100 for implementing content distribution services. The areafor providing communication services is divided Into cells of desiredsize, and base stations ex106, ex107, ex108, ex109, and ex110 which arefixed wireless stations are placed in each of the cells.

The content providing system ex100 is connected to devices, such as acomputer ex111, a personal digital assistant (PDA) ex112, a cameraex113, a cellular phone ex114 and a game machine ex115, via the Internetex101, an Internet service provider ex102, a telephone network ex104, aswell as the base stations ex106 to ex110, respectively.

However, the configuration of the content providing system ex100 is notlimited to the configuration shown in FIG. 15, and a combination inwhich any of the elements are connected is acceptable. In addition, eachdevice may be directly connected to the telephone network ex104, ratherthan via the base stations ex106 to ex110 which are the fixed wirelessstations. Furthermore, the devices may be interconnected to each othervia a short distance wireless communication and others.

The camera ex113, such as a digital video camera, is capable ofcapturing video. A camera ex116, such as a digital camera, is capable ofcapturing both still images and video. Furthermore, the cellular phoneex114 may be the one that meets any of the standards such as GlobalSystem for Mobile Communications (GSM) (registered trademark), CodeDivision Multiple Access (CDMA), Wideband-Code Division Multiple Access(W-CDMA), Long Term Evolution (LTE), and High Speed Packet Access(HSPA). Alternatively, the cellular phone ex114 may be a PersonalHandyphone System (PHS).

In the content providing system ex100, a streaming server ex103 isconnected to the camera ex113 and others via the telephone network ex104and the base station ex109, which enables distribution of images of alive show and others. In such a distribution, a content (for example,video of a music live show) captured by the user using the camera ex113is encoded as described above in each of embodiments (i.e., the camerafunctions as the image encoding apparatus according to an aspect of thepresent disclosure), and the encoded content is transmitted to thestreaming server ex103. On the other hand, the streaming server ex103carries out stream distribution of the transmitted content data to theclients upon their requests. The clients include the computer ex111, thePDA ex112, the camera ex113, the cellular phone ex114, and the gamemachine ex115 that are capable of decoding the above-mentioned encodeddata. Each of the devices that have received the distributed datadecodes and reproduces the encoded data (i.e., functions as the imagedecoding apparatus according to an aspect of the present disclosure).

The captured data may be encoded by the camera ex113 or the streamingserver ex103 that transmits the data, or the encoding processes may beshared between the camera ex113 and the streaming server ex103.Similarly, the distributed data may be decoded by the clients or thestreaming server ex103, or the decoding processes may be shared betweenthe clients and the streaming server ex103. Furthermore, the data of thestill images and video captured by not only the camera ex113 but alsothe camera ex116 may be transmitted to the streaming server ex103through the computer ex111. The encoding processes may be performed bythe camera ex116, the computer ex111, or the streaming server ex103, orshared among them.

Furthermore, the coding processes may be performed by an LSI ex500generally included in each of the computer ex111 and the devices. TheLSI ex500 may be configured of a single chip or a plurality of chips.Software for coding video may be integrated into some type of arecording medium (such as a CD-ROM, a flexible disk, and a hard disk)that is readable by the computer ex111 and others, and the codingprocesses may be performed using the software. Furthermore, when thecellular phone ex114 is equipped with a camera, the video data obtainedby the camera may be transmitted. The video data is data encoded by theLSI ex500 included in the cellular phone ex114.

Furthermore, the streaming server ex103 may be composed of servers andcomputers, and may decentralize data and process the decentralized data,record, or distribute data.

As described above, the clients may receive and reproduce the encodeddata in the content providing system ex100. In other words, the clientscan receive and decode information transmitted by the user, andreproduce the decoded data in real time in the content providing systemex100, so that the user who does not have any particular right andequipment can implement personal broadcasting.

Aside from the example of the content providing system ex100, at leastone of the moving picture coding apparatus (image coding apparatus)described in each of embodiments may be implemented in a digitalbroadcasting system ex200 illustrated in FIG. 16. More specifically, abroadcast station ex201 communicates or transmits, via radio waves to abroadcast satellite ex202, multiplexed data obtained by multiplexingaudio data and others onto video data. The video data is data encoded bythe moving picture encoding method described in each of embodiments(i.e., data encoded by the image encoding apparatus according to anaspect of the present disclosure). Upon receipt of the multiplexed data,the broadcast satellite ex202 transmits radio waves for broadcasting.Then, a home-use antenna ex204 with a satellite broadcast receptionfunction receives the radio waves. Next, a device such as a television(receiver) ex300 and a set top box (STB) ex217 decodes the receivedmultiplexed data, and reproduces the decoded data (i.e., functions asthe image decoding apparatus according to an aspect of the presentdisclosure).

Furthermore, a reader/recorder ex218 (i) reads and decodes themultiplexed data recorded on a recording medium ex215, such as a DVD anda BD, or (i) encodes video signals in the recording medium ex215, and insome cases, writes data obtained by multiplexing an audio signal on theencoded data. The reader/recorder ex218 can include the moving picturedecoding apparatus or the moving picture encoding apparatus as shown ineach of embodiments. In this case, the reproduced video signals aredisplayed on the monitor ex219, and can be reproduced by another deviceor system using the recording medium ex215 on which the multiplexed datais recorded. It is also possible to implement the moving picturedecoding apparatus in the set top box ex217 connected to the cable ex203for a cable television or to the antenna ex204 for satellite and/orterrestrial broadcasting, so as to display the video signals on themonitor ex219 of the television ex300. The moving picture decodingapparatus may be implemented not in the set top box but in thetelevision ex300.

FIG. 17 illustrates the television (receiver) ex300 that uses the movingpicture encoding method and the moving picture decoding method describedin each of embodiments. The television ex300 includes: a tuner ex301that obtains or provides multiplexed data obtained by multiplexing audiodata onto video data, through the antenna ex204 or the cable ex203, etc.that receives a broadcast; a modulation/demodulation unit ex302 thatdemodulates the received multiplexed data or modulates data intomultiplexed data to be supplied outside; and amultiplexing/demultiplexing unit ex303 that demultiplexes the modulatedmultiplexed data into video data and audio data, or multiplexes videodata and audio data encoded by a signal processing unit ex306 into data.

The television ex300 further includes: a signal processing unit ex306including an audio signal processing unit ex304 and a video signalprocessing unit ex305 that code each of audio data and video data,(which function as the image coding apparatus according to the aspectsof the present disclosure); and an output unit ex309 including a speakerex307 that provides the decoded audio signal, and a display unit ex308that displays the decoded video signal, such as a display. Furthermore,the television ex300 includes an interface unit ex317 including anoperation input unit ex312 that receives an input of a user operation.Furthermore, the television ex300 includes a control unit ex310 thatcontrols overall each constituent element of the television ex300, and apower supply circuit unit ex311 that supplies power to each of theelements. Other than the operation input unit ex312, the interface unitex317 may include: a bridge ex313 that is connected to an externaldevice, such as the reader/recorder ex218; a slot unit ex314 forenabling attachment of the recording medium ex216, such as an SD card; adriver ex315 to be connected to an external recording medium, such as ahard disk; and a modem ex316 to be connected to a telephone network.Here, the recording medium ex216 can electrically record informationusing a non-volatile/volatile semiconductor memory element for storage.The constituent elements of the television ex300 are connected to eachother through a synchronous bus.

First, the configuration in which the television ex300 decodesmultiplexed data obtained from outside through the antenna ex204 andothers and reproduces the decoded data will be described. In thetelevision ex300, upon a user operation through a remote controllerex220 and others, the multiplexing/demultiplexing unit ex303demultiplexes the multiplexed data demodulated by themodulation/demodulation unit ex302, under control of the control unitex310 including a CPU. Furthermore, the audio signal processing unitex304 decodes the demultiplexed audio data, and the video signalprocessing unit ex305 decodes the demultiplexed video data, using thedecoding method described in each of embodiments, in the televisionex300. The output unit ex309 provides the decoded video signal and audiosignal outside, respectively. When the output unit ex309 provides thevideo signal and the audio signal, the signals may be temporarily storedin buffers ex318 and ex319, and others so that the signals arereproduced in synchronization with each other. Furthermore, thetelevision ex300 may read multiplexed data not through a broadcast andothers but from the recording media ex215 and ex216, such as a magneticdisk, an optical disk, and a SD card. Next, a configuration in which thetelevision ex300 encodes an audio signal and a video signal, andtransmits the data outside or writes the data on a recording medium willbe described. In the television ex300, upon a user operation through theremote controller ex220 and others, the audio signal processing unitex304 encodes an audio signal, and the video signal processing unitex305 encodes a video signal, under control of the control unit ex310using the encoding method described in each of embodiments. Themultiplexing/demultiplexing unit ex303 multiplexes the encoded videosignal and audio signal, and provides the resulting signal outside. Whenthe multiplexing/demultiplexing unit ex303 multiplexes the video signaland the audio signal, the signals may be temporarily stored in thebuffers ex320 and ex321, and others so that the signals are reproducedin synchronization with each other. Here, the buffers ex318, ex319,ex320, and ex321 may be plural as Illustrated, or at least one buffermay be shared in the television ex300. Furthermore, data may be storedin a buffer so that the system overflow and underflow may be avoidedbetween the modulation/demodulation unit ex302 and themultiplexing/demultiplexing unit ex303, for example.

Furthermore, the television ex300 may include a configuration forreceiving an AV input from a microphone or a camera other than theconfiguration for obtaining audio and video data from a broadcast or arecording medium, and may encode the obtained data. Although thetelevision ex300 can encode, multiplex, and provide outside data in thedescription, it may be capable of only receiving, decoding, andproviding outside data but not the encoding, multiplexing, and providingoutside data.

Furthermore, when the reader/recorder ex218 reads or writes multiplexeddata from or on a recording medium, one of the television ex300 and thereader/recorder ex218 may code the multiplexed data, and the televisionex300 and the reader/recorder ex218 may share the coding partly.

As an example, FIG. 18 illustrates a configuration of an informationreproducing/recording unit ex400 when data is read or written from or onan optical disk. The information reproducing/recording unit ex400includes constituent elements ex401, ex402, ex403, ex404, ex405, ex406,and ex407 to be described hereinafter. The optical head ex401 Irradiatesa laser spot in a recording surface of the recording medium ex215 thatis an optical disk to write information, and detects reflected lightfrom the recording surface of the recording medium ex215 to read theinformation. The modulation recording unit ex402 electrically drives asemiconductor laser included in the optical head ex401, and modulatesthe laser light according to recorded data. The reproductiondemodulating unit ex403 amplifies a reproduction signal obtained byelectrically detecting the reflected light from the recording surfaceusing a photo detector included in the optical head ex401, anddemodulates the reproduction signal by separating a signal componentrecorded on the recording medium ex215 to reproduce the necessaryinformation. The buffer ex404 temporarily holds the information to berecorded on the recording medium ex215 and the information reproducedfrom the recording medium ex215. The disk motor ex405 rotates therecording medium ex215. The servo control unit ex406 moves the opticalhead ex401 to a predetermined information track while controlling therotation drive of the disk motor ex405 so as to follow the laser spot.The system control unit ex407 controls overall the informationreproducing/recording unit ex400. The reading and writing processes canbe implemented by the system control unit ex407 using variousinformation stored in the buffer ex404 and generating and adding newinformation as necessary, and by the modulation recording unit ex402,the reproduction demodulating unit ex403, and the servo control unitex406 that record and reproduce information through the optical headex401 while being operated in a coordinated manner. The system controlunit ex407 includes, for example, a microprocessor, and executesprocessing by causing a computer to execute a program for read andwrite.

Although the optical head ex401 irradiates a laser spot in thedescription, it may perform high-density recording using near fieldlight.

FIG. 19 illustrates the recording medium ex215 that is the optical disk.On the recording surface of the recording medium ex215, guide groovesare spirally formed, and an information track ex230 records, in advance,address information indicating an absolute position on the diskaccording to change in a shape of the guide grooves. The addressinformation includes information for determining positions of recordingblocks ex231 that are a unit for recording data. Reproducing theinformation track ex230 and reading the address information in anapparatus that records and reproduces data can lead to determination ofthe positions of the recording blocks. Furthermore, the recording mediumex215 includes a data recording area ex233, an inner circumference areaex232, and an outer circumference area ex234. The data recording areaex233 is an area for use in recording the user data. The innercircumference area ex232 and the outer circumference area ex234 that areinside and outside of the data recording area ex233, respectively arefor specific use except for recording the user data. The informationreproducing/recording unit 400 reads and writes encoded audio, encodedvideo data, or multiplexed data obtained by multiplexing the encodedaudio and video data, from and on the data recording area ex233 of therecording medium ex215.

Although an optical disk having a layer, such as a DVD and a BD isdescribed as an example in the description, the optical disk is notlimited to such, and may be an optical disk having a multilayerstructure and capable of being recorded on a part other than thesurface. Furthermore, the optical disk may have a structure formultidimensional recording/reproduction, such as recording ofinformation using light of colors with different wavelengths in the sameportion of the optical disk and for recording information havingdifferent layers from various angles.

Furthermore, a car ex210 having an antenna ex205 can receive data fromthe satellite ex202 and others, and reproduce video on a display devicesuch as a car navigation system ex211 set in the car ex210, in thedigital broadcasting system ex200. Here, a configuration of the carnavigation system ex211 will be a configuration, for example, includinga GPS receiving unit from the configuration illustrated in FIG. 17. Thesame will be true for the configuration of the computer ex111, thecellular phone ex114, and others.

FIG. 20A illustrates the cellular phone ex114 that uses the movingpicture coding method described in embodiments. The cellular phone ex114includes: an antenna ex350 for transmitting and receiving radio wavesthrough the base station ex110; a camera unit ex365 capable of capturingmoving and still images; and a display unit ex358 such as a liquidcrystal display for displaying the data such as decoded video capturedby the camera unit ex365 or received by the antenna ex350. The cellularphone ex114 further includes: a main body unit including an operationkey unit ex366; an audio output unit ex357 such as a speaker for outputof audio; an audio input unit ex356 such as a microphone for input ofaudio; a memory unit ex367 for storing captured video or still pictures,recorded audio, coded data of the received video, the still pictures,e-mails, or others; and a slot unit ex364 that is an interface unit fora recording medium that stores data in the same manner as the memoryunit ex367.

Next, an example of a configuration of the cellular phone ex114 will bedescribed with reference to FIG. 20B. In the cellular phone ex114, amain control unit ex360 designed to control overall each unit of themain body including the display unit ex358 as well as the operation keyunit ex366 is connected mutually, via a synchronous bus ex370, to apower supply circuit unit ex361, an operation input control unit ex362,a video signal processing unit ex355, a camera interface unit ex363, aliquid crystal display (LCD) control unit ex359, amodulation/demodulation unit ex352, a multiplexing/demultiplexing unitex353, an audio signal processing unit ex354, the slot unit ex364, andthe memory unit ex367.

When a call-end key or a power key is turned ON by a user's operation,the power supply circuit unit ex361 supplies the respective units withpower from a battery pack so as to activate the cell phone ex114.

In the cellular phone ex114, the audio signal processing unit ex354converts the audio signals collected by the audio Input unit ex356 invoice conversation mode into digital audio signals under the control ofthe main control unit ex360 including a CPU, ROM, and RAM. Then, themodulation/demodulation unit ex352 performs spread spectrum processingon the digital audio signals, and the transmitting and receiving unitex351 performs digital-to-analog conversion and frequency conversion onthe data, so as to transmit the resulting data via the antenna ex350.Also, in the cellular phone ex114, the transmitting and receiving unitex351 amplifies the data received by the antenna ex350 in voiceconversation mode and performs frequency conversion and theanalog-to-digital conversion on the data. Then, themodulation/demodulation unit ex352 performs inverse spread spectrumprocessing on the data, and the audio signal processing unit ex354converts it into analog audio signals, so as to output them via theaudio output unit ex357.

Furthermore, when an e-mail in data communication mode is transmitted,text data of the e-mail inputted by operating the operation key unitex366 and others of the main body is sent out to the main control unitex360 via the operation input control unit ex362. The main control unitex360 causes the modulation/demodulation unit ex352 to perform spreadspectrum processing on the text data, and the transmitting and receivingunit ex351 performs the digital-to-analog conversion and the frequencyconversion on the resulting data to transmit the data to the basestation ex110 via the antenna ex350. When an e-mail is received,processing that is approximately inverse to the processing fortransmitting an e-mail is performed on the received data, and theresulting data is provided to the display unit ex358.

When video, still images, or video and audio in data communication modeis or are transmitted, the video signal processing unit ex355 compressesand encodes video signals supplied from the camera unit ex365 using themoving picture encoding method shown in each of embodiments (i.e.,functions as the image encoding apparatus according to the aspect of thepresent disclosure), and transmits the encoded video data to themultiplexing/demultiplexing unit ex353. In contrast, during when thecamera unit ex365 captures video, still images, and others, the audiosignal processing unit ex354 encodes audio signals collected by theaudio input unit ex356, and transmits the encoded audio data to themultiplexing/demultiplexing unit ex353.

The multiplexing/demultiplexing unit ex353 multiplexes the encoded videodata supplied from the video signal processing unit ex355 and theencoded audio data supplied from the audio signal processing unit ex354,using a predetermined method. Then, the modulation/demodulation unit(modulation/demodulation circuit unit) ex352 performs spread spectrumprocessing on the multiplexed data, and the transmitting and receivingunit ex351 performs digital-to-analog conversion and frequencyconversion on the data so as to transmit the resulting data via theantenna ex350.

When receiving data of a video file which is linked to a Web page andothers in data communication mode or when receiving an e-mail with videoand/or audio attached, in order to decode the multiplexed data receivedvia the antenna ex350, the multiplexing/demultiplexing unit ex353demultiplexes the multiplexed data into a video data bit stream and anaudio data bit stream, and supplies the video signal processing unitex355 with the encoded video data and the audio signal processing unitex354 with the encoded audio data, through the synchronous bus ex370.The video signal processing unit ex355 decodes the video signal using amoving picture decoding method corresponding to the moving pictureencoding method shown in each of embodiments (i.e., functions as theimage decoding apparatus according to the aspect of the presentdisclosure), and then the display unit ex358 displays, for instance, thevideo and still images included in the video file linked to the Web pagevia the LCD control unit ex359. Furthermore, the audio signal processingunit ex354 decodes the audio signal, and the audio output unit ex357provides the audio.

Furthermore, similarly to the television ex300, a terminal such as thecellular phone ex114 probably have 3 types of implementationconfigurations including not only (i) a transmitting and receivingterminal including both an encoding apparatus and a decoding apparatus,but also (ii) a transmitting terminal including only an encodingapparatus and (iii) a receiving terminal including only a decodingapparatus. Although the digital broadcasting system ex200 receives andtransmits the multiplexed data obtained by multiplexing audio data ontovideo data in the description, the multiplexed data may be data obtainedby multiplexing not audio data but character data related to video ontovideo data, and may be not multiplexed data but video data itself.

As such, the moving picture coding method in each of embodiments can beused in any of the devices and systems described. Thus, the advantagesdescribed in each of embodiments can be obtained.

Furthermore, various modifications and revisions can be made in any ofthe embodiments in the present disclosure.

Embodiment 5

Video data can be generated by switching, as necessary, between (i) themoving picture encoding method or the moving picture encoding apparatusshown in each of embodiments and (ii) a moving picture encoding methodor a moving picture encoding apparatus in conformity with a differentstandard, such as MPEG-2, MPEG-4 AVC, and VC-1.

Here, when a plurality of video data that conforms to the differentstandards is generated and is then decoded, the decoding methods need tobe selected to conform to the different standards. However, since towhich standard each of the plurality of the video data to be decodedconforms cannot be detected, an appropriate decoding method cannot beselected.

In view of this, multiplexed data obtained by multiplexing audio dataand others onto video data has a structure including identificationinformation indicating to which standard the video data conforms. Thespecific structure of the multiplexed data including the video datagenerated in the moving picture encoding method and by the movingpicture encoding apparatus shown in each of embodiments will behereinafter described. The multiplexed data is a digital stream in theMPEG-2 Transport Stream format.

FIG. 21 illustrates a structure of the multiplexed data. As illustratedin FIG. 21, the multiplexed data can be obtained by multiplexing atleast one of a video stream, an audio stream, a presentation graphicsstream (PG), and an interactive graphics stream. The video streamrepresents primary video and secondary video of a movie, the audiostream (IG) represents a primary audio part and a secondary audio partto be mixed with the primary audio part, and the presentation graphicsstream represents subtitles of the movie. Here, the primary video isnormal video to be displayed on a screen, and the secondary video isvideo to be displayed on a smaller window in the primary video.Furthermore, the interactive graphics stream represents an interactivescreen to be generated by arranging the GUI components on a screen. Thevideo stream is encoded in the moving picture encoding method or by themoving picture encoding apparatus shown in each of embodiments, or in amoving picture encoding method or by a moving picture encoding apparatusin conformity with a conventional standard, such as MPEG-2, MPEG-4 AVC,and VC-1. The audio stream is encoded in accordance with a standard,such as Dolby-AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD, and linearPCM.

Each stream included in the multiplexed data is identified by PID. Forexample, 0x1011 is allocated to the video stream to be used for video ofa movie, 0x1100 to 0x111F are allocated to the audio streams, 0x1200 to0x121F are allocated to the presentation graphics streams, 0x1400 to0x141F are allocated to the interactive graphics streams, 0x1B00 to0x1B1F are allocated to the video streams to be used for secondary videoof the movie, and 0x1A00 to 0x1A1F are allocated to the audio streams tobe used for the secondary audio to be mixed with the primary audio.

FIG. 22 schematically illustrates how data is multiplexed. First, avideo stream ex235 composed of video frames and an audio stream ex238composed of audio frames are transformed into a stream of PES packetsex236 and a stream of PES packets ex239, and further into TS packetsex237 and TS packets ex240, respectively. Similarly, data of apresentation graphics stream ex241 and data of an interactive graphicsstream ex244 are transformed into a stream of PES packets ex242 and astream of PES packets ex245, and further into TS packets ex243 and TSpackets ex246, respectively. These TS packets are multiplexed into astream to obtain multiplexed data ex247.

FIG. 23 illustrates how a video stream is stored in a stream of PESpackets in more detail. The first bar in FIG. 23 shows a video framestream in a video stream. The second bar shows the stream of PESpackets. As indicated by arrows denoted as yy1, yy2, yy3, and yy4 inFIG. 23, the video stream is divided into pictures as I pictures, Bpictures, and P pictures each of which is a video presentation unit, andthe pictures are stored in a payload of each of the PES packets. Each ofthe PES packets has a PES header, and the PES header stores aPresentation Time-Stamp (PTS) indicating a display time of the picture,and a Decoding Time-Stamp (DTS) indicating a decoding time of thepicture.

FIG. 24 illustrates a format of TS packets to be finally written on themultiplexed data. Each of the TS packets is a 188-byte fixed lengthpacket including a 4-byte TS header having information, such as a PIDfor identifying a stream and a 184-byte TS payload for storing data. ThePES packets are divided, and stored in the TS payloads, respectively.When a BD ROM is used, each of the TS packets is given a 4-byteTP_Extra_Header, thus resulting in 192-byte source packets. The sourcepackets are written on the multiplexed data. The TP_Extra_Header storesinformation such as an Arrival_Time_Stamp (ATS). The ATS shows atransfer start time at which each of the TS packets is to be transferredto a PID filter. The source packets are arranged in the multiplexed dataas shown at the bottom of FIG. 24. The numbers incrementing from thehead of the multiplexed data are called source packet numbers (SPNs).

Each of the TS packets included in the multiplexed data includes notonly streams of audio, video, subtitles and others, but also a ProgramAssociation Table (PAT), a Program Map Table (PMT), and a Program ClockReference (PCR). The PAT shows what a PID in a PMT used in themultiplexed data indicates, and a PID of the PAT itself is registered aszero. The PMT stores PIDs of the streams of video, audio, subtitles andothers included in the multiplexed data, and attribute information ofthe streams corresponding to the PIDs. The PMT also has variousdescriptors relating to the multiplexed data. The descriptors haveinformation such as copy control information showing whether copying ofthe multiplexed data is permitted or not. The PCR stores STC timeinformation corresponding to an ATS showing when the PCR packet istransferred to a decoder, in order to achieve synchronization between anArrival Time Clock (ATC) that is a time axis of ATSs, and an System TimeClock (STC) that is a time axis of PTSs and DTSs.

FIG. 25 illustrates the data structure of the PMT in detail. A PMTheader is disposed at the top of the PMT. The PMT header describes thelength of data included in the PMT and others. A plurality ofdescriptors relating to the multiplexed data is disposed after the PMTheader. Information such as the copy control information is described inthe descriptors. After the descriptors, a plurality of pieces of streaminformation relating to the streams included in the multiplexed data isdisposed. Each piece of stream information includes stream descriptorseach describing information, such as a stream type for identifying acompression codec of a stream, a stream PID, and stream attributeinformation (such as a frame rate or an aspect ratio). The streamdescriptors are equal in number to the number of streams in themultiplexed data.

When the multiplexed data is recorded on a recording medium and others,it is recorded together with multiplexed data information files.

Each of the multiplexed data information files is management informationof the multiplexed data as shown in FIG. 26. The multiplexed datainformation files are in one to one correspondence with the multiplexeddata, and each of the files includes multiplexed data information,stream attribute information, and an entry map.

As Illustrated in FIG. 26, the multiplexed data information includes asystem rate, a reproduction start time, and a reproduction end time. Thesystem rate indicates the maximum transfer rate at which a system targetdecoder to be described later transfers the multiplexed data to a PIDfilter. The intervals of the ATSs included in the multiplexed data areset to not higher than a system rate. The reproduction start timeindicates a PTS in a video frame at the head of the multiplexed data. Aninterval of one frame is added to a PTS in a video frame at the end ofthe multiplexed data, and the PTS is set to the reproduction end time.

As shown in FIG. 27, a piece of attribute information is registered inthe stream attribute information, for each PID of each stream includedin the multiplexed data. Each piece of attribute information hasdifferent information depending on whether the corresponding stream is avideo stream, an audio stream, a presentation graphics stream, or aninteractive graphics stream. Each piece of video stream attributeinformation carries information including what kind of compression codecis used for compressing the video stream, and the resolution, aspectratio and frame rate of the pieces of picture data that is included inthe video stream. Each piece of audio stream attribute informationcarries information including what kind of compression codec is used forcompressing the audio stream, how many channels are included in theaudio stream, which language the audio stream supports, and how high thesampling frequency is. The video stream attribute information and theaudio stream attribute information are used for initialization of adecoder before the player plays back the information.

In the present embodiment, the multiplexed data to be used is of astream type included in the PMT. Furthermore, when the multiplexed datais recorded on a recording medium, the video stream attributeinformation included in the multiplexed data information is used. Morespecifically, the moving picture encoding method or the moving pictureencoding apparatus described in each of embodiments includes a step or aunit for allocating unique information indicating video data generatedby the moving picture encoding method or the moving picture encodingapparatus in each of embodiments, to the stream type included in the PMTor the video stream attribute information. With the configuration, thevideo data generated by the moving picture encoding method or the movingpicture encoding apparatus described in each of embodiments can bedistinguished from video data that conforms to another standard.

Furthermore, FIG. 28 illustrates steps of the moving picture decodingmethod according to the present embodiment. In Step exS100, the streamtype included in the PMT or the video stream attribute informationincluded in the multiplexed data information is obtained from themultiplexed data. Next, in Step exS101, it is determined whether or notthe stream type or the video stream attribute information indicates thatthe multiplexed data is generated by the moving picture encoding methodor the moving picture encoding apparatus in each of embodiments. When itis determined that the stream type or the video stream attributeinformation indicates that the multiplexed data is generated by themoving picture encoding method or the moving picture encoding apparatusin each of embodiments, in Step exS102, decoding is performed by themoving picture decoding method in each of embodiments. Furthermore, whenthe stream type or the video stream attribute information indicatesconformance to the conventional standards, such as MPEG-2, MPEG-4 AVC,and VC-1, in Step exS103, decoding is performed by a moving picturedecoding method in conformity with the conventional standards.

As such, allocating a new unique value to the stream type or the videostream attribute information enables determination whether or not themoving picture decoding method or the moving picture decoding apparatusthat is described in each of embodiments can perform decoding. Even whenmultiplexed data that conforms to a different standard is input, anappropriate decoding method or apparatus can be selected. Thus, itbecomes possible to decode information without any error. Furthermore,the moving picture encoding method or apparatus, or the moving picturedecoding method or apparatus in the present embodiment can be used inthe devices and systems described above.

Embodiment 6

Each of the moving picture coding method and the moving picture codingapparatus in each of embodiments is typically achieved in the form of anintegrated circuit or a Large Scale Integrated (LSI) circuit. As anexample of the LSI, FIG. 29 illustrates a configuration of the LSI ex500that is made into one chip. The LSI ex500 includes elements ex501,ex502, ex503, ex504, ex505, ex506, ex507, ex508, and ex509 to bedescribed below, and the elements are connected to each other through abus ex510. The power supply circuit unit ex505 is activated by supplyingeach of the elements with power when the power supply circuit unit ex505is turned on.

For example, when encoding is performed, the LSI ex500 receives an AVsignal from a microphone ex117, a camera ex113, and others through an AV10 ex509 under control of a control unit ex501 including a CPU ex502, amemory controller ex503, a stream controller ex504, and a drivingfrequency control unit ex512. The received AV signal is temporarilystored in an external memory ex511, such as an SDRAM. Under control ofthe control unit ex501, the stored data is segmented into data portionsaccording to the processing amount and speed to be transmitted to asignal processing unit ex507. Then, the signal processing unit ex507encodes an audio signal and/or a video signal. Here, the encoding of thevideo signal is the encoding described in each of embodiments.Furthermore, the signal processing unit ex507 sometimes multiplexes theencoded audio data and the encoded video data, and a stream IO ex506provides the multiplexed data outside. The provided multiplexed data istransmitted to the base station ex107, or written on the recordingmedium ex215. When data sets are multiplexed, the data should betemporarily stored in the buffer ex508 so that the data sets aresynchronized with each other.

Although the memory ex511 is an element outside the LSI ex500, it may beincluded in the LSI ex500. The buffer ex508 is not limited to onebuffer, but may be composed of buffers. Furthermore, the LSI ex500 maybe made into one chip or a plurality of chips.

Furthermore, although the control unit ex501 includes the CPU ex502, thememory controller ex503, the stream controller ex504, the drivingfrequency control unit ex512, the configuration of the control unitex501 is not limited to such. For example, the signal processing unitex507 may further include a CPU. Inclusion of another CPU in the signalprocessing unit ex507 can improve the processing speed. Furthermore, asanother example, the CPU ex502 may serve as or be a part of the signalprocessing unit ex507, and, for example, may include an audio signalprocessing unit. In such a case, the control unit ex501 includes thesignal processing unit ex507 or the CPU ex502 including a part of thesignal processing unit ex507.

The name used here is LSI, but it may also be called IC, system LSI,super LSI, or ultra LSI depending on the degree of integration.

Moreover, ways to achieve integration are not limited to the LSI, and aspecial circuit or a general purpose processor and so forth can alsoachieve the Integration. Field Programmable Gate Array (FPGA) that canbe programmed after manufacturing LSIs or a reconfigurable processorthat allows re-configuration of the connection or configuration of anLSI can be used for the same purpose. Such a programmable logic devicecan typically execute the moving picture coding method according to anyof the above embodiments by loading or reading from a memory or the likeone or more programs that are included in software or firmware.

In the future, with advancement in semiconductor technology, a brand-newtechnology may replace LSI. The functional blocks can be integratedusing such a technology. The possibility is that the present disclosureis applied to biotechnology.

Embodiment 7

When video data generated in the moving picture encoding method or bythe moving picture encoding apparatus described in each of embodimentsis decoded, compared to when video data that conforms to a conventionalstandard, such as MPEG-2, MPEG-4 AVC, and VC-1 is decoded, theprocessing amount probably increases. Thus, the LSI ex500 needs to beset to a driving frequency higher than that of the CPU ex502 to be usedwhen video data in conformity with the conventional standard is decoded.However, there is a problem that the power consumption increases whenthe driving frequency is set higher.

In view of this, the moving picture decoding apparatus, such as thetelevision ex300 and the LSI ex500 is configured to determine to whichstandard the video data conforms, and switch between the drivingfrequencies according to the determined standard. FIG. 30 illustrates aconfiguration ex800 in the present embodiment. A driving frequencyswitching unit ex803 sets a driving frequency to a higher drivingfrequency when video data is generated by the moving picture encodingmethod or the moving picture encoding apparatus described in each ofembodiments. Then, the driving frequency switching unit ex803 instructsa decoding processing unit ex801 that executes the moving picturedecoding method described in each of embodiments to decode the videodata. When the video data conforms to the conventional standard, thedriving frequency switching unit ex803 sets a driving frequency to alower driving frequency than that of the video data generated by themoving picture encoding method or the moving picture encoding apparatusdescribed in each of embodiments. Then, the driving frequency switchingunit ex803 instructs the decoding processing unit ex802 that conforms tothe conventional standard to decode the video data.

More specifically, the driving frequency switching unit ex803 includesthe CPU ex502 and the driving frequency control unit ex512 in FIG. 29.Here, each of the decoding processing unit ex801 that executes themoving picture decoding method described in each of embodiments and thedecoding processing unit ex802 that conforms to the conventionalstandard corresponds to the signal processing unit ex507 in FIG. 29. TheCPU ex502 determines to which standard the video data conforms. Then,the driving frequency control unit ex512 determines a driving frequencybased on a signal from the CPU ex502. Furthermore, the signal processingunit ex507 decodes the video data based on the signal from the CPUex502. For example, the identification information described inEmbodiment 5 is probably used for identifying the video data. Theidentification information is not limited to the one described inEmbodiment 5 but may be any information as long as the informationindicates to which standard the video data conforms. For example, whenwhich standard video data conforms to can be determined based on anexternal signal for determining that the video data is used for atelevision or a disk, etc., the determination may be made based on suchan external signal. Furthermore, the CPU ex502 selects a drivingfrequency based on, for example, a look-up table in which the standardsof the video data are associated with the driving frequencies as shownin FIG. 32. The driving frequency can be selected by storing the look-uptable in the buffer ex508 and in an internal memory of an LSI, and withreference to the look-up table by the CPU ex502.

FIG. 31 illustrates steps for executing a method in the presentembodiment. First, in Step exS200, the signal processing unit ex507obtains identification information from the multiplexed data. Next, inStep exS201, the CPU ex502 determines whether or not the video data isgenerated by the encoding method and the encoding apparatus described ineach of embodiments, based on the identification information. When thevideo data is generated by the moving picture encoding method and themoving picture encoding apparatus described in each of embodiments, inStep exS202, the CPU ex502 transmits a signal for setting the drivingfrequency to a higher driving frequency to the driving frequency controlunit ex512. Then, the driving frequency control unit ex512 sets thedriving frequency to the higher driving frequency. On the other hand,when the identification information indicates that the video dataconforms to the conventional standard, such as MPEG-2, MPEG-4 AVC, andVC-1, in Step exS203, the CPU ex502 transmits a signal for setting thedriving frequency to a lower driving frequency to the driving frequencycontrol unit ex512. Then, the driving frequency control unit ex512 setsthe driving frequency to the lower driving frequency than that in thecase where the video data is generated by the moving picture encodingmethod and the moving picture encoding apparatus described in each ofembodiment.

Furthermore, along with the switching of the driving frequencies, thepower conservation effect can be improved by changing the voltage to beapplied to the LSI ex500 or an apparatus including the LSI ex500. Forexample, when the driving frequency is set lower, the voltage to beapplied to the LSI ex500 or the apparatus including the LSI ex500 isprobably set to a voltage lower than that in the case where the drivingfrequency is set higher.

Furthermore, when the processing amount for decoding is larger, thedriving frequency may be set higher, and when the processing amount fordecoding is smaller, the driving frequency may be set lower as themethod for setting the driving frequency. Thus, the setting method isnot limited to the ones described above. For example, when theprocessing amount for decoding video data in conformity with MPEG-4 AVCis larger than the processing amount for decoding video data generatedby the moving picture encoding method and the moving picture encodingapparatus described in each of embodiments, the driving frequency isprobably set in reverse order to the setting described above.

Furthermore, the method for setting the driving frequency is not limitedto the method for setting the driving frequency lower. For example, whenthe identification information indicates that the video data isgenerated by the moving picture encoding method and the moving pictureencoding apparatus described in each of embodiments, the voltage to beapplied to the LSI ex500 or the apparatus including the LSI ex500 isprobably set higher. When the identification information indicates thatthe video data conforms to the conventional standard, such as MPEG-2,MPEG-4 AVC, and VC-1, the voltage to be applied to the LSI ex500 or theapparatus including the LSI ex500 is probably set lower. As anotherexample, when the identification information indicates that the videodata is generated by the moving picture encoding method and the movingpicture encoding apparatus described in each of embodiments, the drivingof the CPU ex502 does not probably have to be suspended. When theidentification information indicates that the video data conforms to theconventional standard, such as MPEG-2, MPEG-4 AVC, and VC-1, the drivingof the CPU ex502 is probably suspended at a given time because the CPUex502 has extra processing capacity. Even when the identificationinformation indicates that the video data is generated by the movingpicture encoding method and the moving picture encoding apparatusdescribed in each of embodiments, in the case where the CPU ex502 hasextra processing capacity, the driving of the CPU ex502 is probablysuspended at a given time. In such a case, the suspending time isprobably set shorter than that in the case where when the identificationinformation indicates that the video data conforms to the conventionalstandard, such as MPEG-2, MPEG-4 AVC, and VC-1.

Accordingly, the power conservation effect can be improved by switchingbetween the driving frequencies in accordance with the standard to whichthe video data conforms. Furthermore, when the LSI ex500 or theapparatus including the LSI ex500 is driven using a battery, the batterylife can be extended with the power conservation effect.

Embodiment 8

There are cases where a plurality of video data that conforms todifferent standards, is provided to the devices and systems, such as atelevision and a cellular phone. In order to enable decoding theplurality of video data that conforms to the different standards, thesignal processing unit ex507 of the LSI ex500 needs to conform to thedifferent standards. However, increase in the scale of the circuit ofthe LSI ex500 and increase in the cost arise with the individual use ofthe signal processing units ex507 that conform to the respectivestandards.

In view of this, what is conceived is a configuration in which thedecoding processing unit for implementing the moving picture decodingmethod described in each of embodiments and the decoding processing unitthat conforms to the conventional standard, such as MPEG-2, MPEG-4 AVC,and VC-1 are partly shared. Ex900 in FIG. 33A shows an example of theconfiguration. For example, the moving picture decoding method describedin each of embodiments and the moving picture decoding method thatconforms to MPEG-4 AVC have, partly in common, the details ofprocessing, such as entropy encoding, inverse quantization, deblockingfiltering, and motion compensated prediction. The details of processingto be shared probably include use of a decoding processing unit ex902that conforms to MPEG-4 AVC. In contrast, a dedicated decodingprocessing unit ex901 is probably used for other processing which isunique to an aspect of the present disclosure and does not conform toMPEG-4 AVC. Since the aspect of the present disclosure is characterizedby filtering in particular, for example, the dedicated decodingprocessing unit ex901 is used for filtering. Otherwise, the decodingprocessing unit is probably shared for one of the entropy decoding,inverse quantization, and motion compensation, or all of the processing.The decoding processing unit for implementing the moving picturedecoding method described in each of embodiments may be shared for theprocessing to be shared, and a dedicated decoding processing unit may beused for processing unique to that of MPEG-4 AVC.

Furthermore, ex1000 in FIG. 33B shows another example in that processingis partly shared. This example uses a configuration including adedicated decoding processing unit ex1001 that supports the processingunique to an aspect of the present disclosure, a dedicated decodingprocessing unit ex1002 that supports the processing unique to anotherconventional standard, and a decoding processing unit ex1003 thatsupports processing to be shared between the moving picture decodingmethod according to the aspect of the present disclosure and theconventional moving picture decoding method. Here, the dedicateddecoding processing units ex1001 and ex1002 are not necessarilyspecialized for the processing according to the aspect of the presentdisclosure and the processing of the conventional standard,respectively, and may be the ones capable of implementing generalprocessing. Furthermore, the configuration of the present embodiment canbe implemented by the LSI ex500.

As such, reducing the scale of the circuit of an LSI and reducing thecost are possible by sharing the decoding processing unit for theprocessing to be shared between the moving picture decoding methodaccording to the aspect of the present disclosure and the moving picturedecoding method in conformity with the conventional standard.

Although only some exemplary embodiments of the present invention havebeen described in detail above, those skilled in the art will readilyappreciate that many modifications are possible in the exemplaryembodiments without materially departing from the novel teachings andadvantages of the present invention. Accordingly, all such modificationsare intended to be included within the scope of the present invention.

INDUSTRIAL APPLICABILITY

The image encoding method for encoding code streams which facilitateparallel decoding processes according to an aspect of the presentdisclosure and the image decoding method for decoding the encodedstreams make it possible to suppress distortions which occur betweenimage areas subjected to the parallel decoding processes. These encodingmethod and decoding method make it possible to implement, at low cost,an image encoding device and an image decoding device capable ofperforming high-speed operation, and thus are applicable to variousapplications such as storage, transmission, communication, etc. Forexample, the image encoding device and the image decoding device areapplicable to high-resolution information display devices and imagingdevices such as television receivers, digital video recorders, carnavigation systems, mobile phones, digital cameras, digital videocameras, etc, and thus are highly applicable.

1. An image decoding method for decoding an image from a bitstream on aper block basis, the image decoding method comprising: predicting acurrent block in the image using a reference block different from thecurrent block, to generate a prediction block; and generating areconstructed block using the prediction block, wherein the generatingincludes: first filtering for filtering a boundary between thereconstructed block and a decoded neighboring block neighboring thecurrent block, using a first filter strength which is set using firstprediction information for the prediction of the current block and 15second prediction information for prediction of the decoded neighboringblock; second filtering for filtering the boundary using a second filterstrength which is set without using the first prediction information andthe second prediction information; and switching whether or not toexecute the second filtering, based on the boundary, wherein the firstfiltering is in-loop filtering in a loop in which a filteredreconstructed block is used as a reference block for another block, andthe second filtering is post filtering outside the loop.
 2. The imagedecoding method according to claim 1, wherein the switching includes:(i) performing the second filtering without performing the firstfiltering when the boundary is a first boundary; and (ii) performing thefirst filtering without performing the second filtering when theboundary is not the first boundary.
 3. The image decoding methodaccording to claim 1, wherein the second filter strength is set as apredetermined fixed value.
 4. The image decoding method according toclaim 1, wherein the second filter strength is set based on supplementalinformation included in a header of the bitstream.
 5. The image decodingmethod according to claim 4, wherein the generating further includesdetermining whether or not the boundary is a first boundary, based onthe supplemental Information.
 6. The image decoding method according toclaim 1, wherein in the predicting, the reconstructed block on which thesecond filtering is not performed is used when the current block is usedto predict another block.
 7. An image decoding device comprising:processing circuitry; and storage which is coupled to the processingcircuitry, wherein the processing circuitry executes decoding on animage on a per block basis from a bitstream, the decoding includes:predicting a current block in the image using a reference blockdifferent from the current block, to generate a prediction block, thereference block being in the storage; and generating a reconstructedblock using the prediction block, and wherein the generating includes:first filtering for filtering a boundary between the reconstructed blockand a decoded neighboring block neighboring the current block, using afirst filter strength which is set using first prediction informationfor the prediction of the current block and second predictioninformation for prediction of the decoded neighboring block; secondfiltering for filtering the boundary using a second filter strengthwhich is set without using the first prediction information and thesecond prediction information; and switching whether or not to executethe second filtering, based on the boundary, wherein the first filteringis in-loop filtering in a loop in which a filtered reconstructed blockis used as a reference block for another block, and the second filteringis post filtering outside the loop.
 8. An image encoding method forencoding an image on a per block basis, the image encoding methodcomprising: predicting a current block included in the image using areference block different from the current block, to generate aprediction block; encoding the current block using the prediction block;and generating a reconstructed block from the current block, wherein thegenerating includes: first filtering for filtering a boundary betweenthe reconstructed block and a reconstructed neighboring block generatedfrom an encoded neighboring block neighboring the current block, using afirst filter strength which is set using first prediction informationfor the prediction of the current block and second predictioninformation for prediction of the neighboring block; second filteringfor filtering the boundary, using a second filter strength which is setwithout using the first prediction information and the second predictioninformation; and switching whether or not to execute the secondfiltering, based on the boundary, wherein the first filtering is in-loopfiltering in a loop in which a filtered reconstructed block is used as areference block for another block, and the second filtering is postfiltering outside the loop.
 9. The image encoding method according toclaim 8, wherein the switching includes: (i) performing the secondfiltering without performing the first filtering when the boundary is afirst boundary; and (ii) performing the first filtering withoutperforming the second filtering when the boundary is not the firstboundary.
 10. The image encoding method according to claim 8, whereinthe second filter strength is set as a predetermined fixed value. 11.The image encoding method according to claim 8, wherein the secondfilter strength is set based on supplemental information included in aheader of the bitstream.
 12. The image encoding method according toclaim 11, wherein the supplemental information includes information fordetermining whether or not the boundary is a first boundary.
 13. Theimage encoding method according to claim 8, wherein in the predicting,the reconstructed block on which the second filtering is not performedis used when the current block is used to predict another block.
 14. Animage encoding device comprising: processing circuitry; and storagewhich is coupled to the processing circuitry, wherein the processingcircuitry executes encoding on an image on a per block basis, theencoding includes: predicting a current block included in the imageusing a reference block different from the current block, to generate aprediction block; encoding the current block using the prediction block;and generating a reconstructed block from the current block, wherein thegenerating includes: first filtering for filtering a boundary betweenthe reconstructed block and a reconstructed neighboring block generatedfrom an encoded neighboring block neighboring the current block, using afirst filter strength which is set using prediction information used inthe prediction of the current block and prediction information used inprediction of the neighboring block; second filtering for filtering theboundary, using a second filter strength which is set without using theprediction information of the current block and the predictioninformation of the encoded neighboring block; and switching whether ornot to execute the second filtering, based on the boundary, wherein thefirst filtering is in-loop filtering in a loop in which a filteredreconstructed block is used as a reference block for the other block,and the second filtering is post filtering outside a loop in which afiltered reconstructed block is not used as the reference block for theother block.